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Abstract 

For W a finite Coxeter group, a formula is found for the size of W equivalence classes of 
subsets of a base. The proof is a case-by-case analysis using results and tables of Orlik and 
Solomon. As a corollary we obtain an alternating sum identity which generalizes a well-known 
identity from the theory of Coxeter groups. 



1 Introduction 

Let W be a finite Coxeter group and n a set of fundamental roots or base of W . Note that W 
acts on subsets of n. For J, K C n, one writes J ~ K if J = w{K) for some w € W. Letting 
\(K) denote the equivalence class of K under this action, a natural problem is to compute the 
size |A(i^)| of this equivalence class. It suffices to work with irreducible W. For the classical 
types |A(if)| can be computed without much difficultly; for the exceptional types |A(if)| has been 
essentially computed by Carter Q. 

Theorem Q of this note will give a formula for |A(iT)| in terms of the index of the parabolic 
subgroup Wk in its normalizer N\y(Wk) and the characteristic polynomial x(£ , t) of an upper 
interval in the intersection lattice L of root hyperplanes corresponding to W. This result is of 
interest in that it gives an expression for |A(X)| in terms of quantities naturally associated to W 
and K. Our proof is a case-by-case analysis; a unified proof for all finite Coxeter groups would be 
desirable. 

Theorem ^ applies Theorem |l] to obtain a generalization of the alternating sum formula 

y ( _ifiM = 1 . 

The above alternating sum formula is discussed in Sections 1.15-1.16 of Humphreys ||. Topo- 
logical proofs using the Hopf trace formula are given by Solomon Q and Steinberg ||. Chapter 
3 of Humphreys gives applications to the invariant theory of Coxeter groups, and Section 6.2 of 
Carter ]2| gives an application to proving the irreducibility of the Steinberg character. We hope 
that our generalization will have similar interpretations and applications. 
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2 Notation 



We collect some notation which will be indispensable for what follows. Denote the set of root 
hyperplanes of W by A. Let L be the set of intersections of the hyperplanes in A, taking V G L. 
Partially order L by reverse inclusion. Recall that the Moebius function fi is defined by fJ,(X, X) = 1 
and J2x<z<Y M-^ Y) = if X < Y and (J,(X, Y) = otherwise. The characteristic polynomial of 
L is defined as 

X (L,x)= Y.^X)x dim{X) - 
xeL 

For K C n, let Fix(Wx) be the fixed space of the parabolic subgroup Wk in its action on V . 
Let L Ftx ( w i<) denote the poset isomorphic to the segment {Y € L(*4,)|Y > Fix(Wk)}- 

Let n be the rank of W, and let Nw(Wk) be the normalizer of Wk in W. As in the introduction, 
for J,KCI1 write J ~ K if J = w(K) for some w £ W and let A(-KT) denote the equivalence class 
of X under this action. 



3 Main Results 

The following lemma of Bergeron, Bergeron, Howlett, and Taylor translates the equivalence relation 
~ into a condition about conjugacy of parabolic subgroups. 



Lemma 1 (Bergeron, Bergeron, Howlett, Taylor fcjy) If J, K Q U, then J ~ K if and only ifWj 
and Wk are conjugate. 

Now we derive an expression for |A(-fT)| in terms of quantities naturally associated to W. 

Theorem 1 Let W be a finite Coxeter group of rank n. Then for all KCII, 

Proof: As both sides of the conjectured equation are multiplicative with respect to direct product 
of groups, it suffices to prove the theorem for irreducible W. This will be done case by case. 

For W of type A n ^i, it is easy to see that J ~ K exactly when Wj and Wk are isomorphic. 
Given K, define equivalence classes called blocks on the set {1, • • • ,n} by letting p ~ q if it is 
possible to transpose p and q using only elements in Wk- Let be the number of blocks of size i. 
A subset JCII such that Wj is isomorphic to Wk arises from any of the (n — J2 i n iV- permutations 
of the blocks of K. One must then divide by FJ^ n^! since permuting the blocks of size i amongst 
themselves leads to the same J. Thus, 



|A(JOI 



(n - J2i rn 
(n-\K\)l 



Win 

h], or 

2.1 of Orlik and Solomon § states that x {L Fix{WK \ t) = (t - 1) • • • (i - (n - \K\ - 1)). Thus, 



Since N<w(Wk) is isomorphic to IliK^j)^ x ^njj one has that \n^(w k )\ = Y[ .n \ ' P ro P os itio n 
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( ,\n-\K\ \W K \ (T Fix{w K ) , _ (n-\K\)\ 
1 J \N W (W K )\ X{L ' ij n,n 2 ! • 

Next we prove the result for type B n (the proof for type C n is identical). It is readily checked that 
K ~ J exactly when Wk and Wj are isomorphic. Let Wk be isomorphic to x A\ 1 -i x ^a 2 -i x " " " 
and let nj be the number of parts of A of size i. Arguing as for type A, 



(re- \K\)\ 

On page 12 of Carter H, it is proved that \ n^w k ) \ = 2 n in-\ ' P ro P os iti° n 2.2 of Orlik and 

Solomon § states that x{L Fix{Wli \t) = (t-l)(t-3) • • • (t- (2(n- |K|) - 1). Therefore we conclude 
as desired that 

\Wk\ ., T Fi*W^ ^ {n-\K\)\ 



( i\n-\K\ \WK\ (T Fix(W K ) ,n 
( 1} \N W (W K )\ X{L ' 1} Yl.ni 



Next we proceed to type D n . Take II = {a%, •••,«„} a fundamental set of simple roots with 

OL\ = £\ — E2, «2 = ^2 — £3) • • • , «n-l = ~ £ni a n = £71-1 + En- 

There are two subcases. The first subcase is that { a n -i,a n } C K. Let be isomorphic to 
Dj x -A^-i x A\ 2 -\ x • • • (here we insist that j + £^ Aj = n) and let be the number of parts of 
A of size i. Arguing as in type A shows that |A(if)| = 2 ^r ^ K ]^' . The extra factor of 2 comes from 

1 Li ni ' 

the symmetry of the Dynkin diagram which switches a n _i and a n but leaves the other roots fixed. 

On page 11 of Carter ||, it is proved that \Nw(W K )\ = YJTwHnA ' P ro P os iti°n 2.6 of Orlik and 
Solomon @ states that X {L Fix( - WK \t) = (t - l)(t - 3) • • • (i -*(2(n - |iT|) - 1). Thus 

1^1 ., r ^) ^ 2(n-|JC|)! 



The second subcase is that {a n _i, a n } <2 if- All J such that J ~ K also satisfy {a n _i, a n } % J. 
Given K, define equivalence classes called blocks on the set {1, • • • , n} by letting p ~ q if it is possible 
to transpose p and q up to sign change, using only elements in Wk- Let rii be the number of blocks 
of size i. Observe that 



\\{K)\ = \{J: J ~K, J C {ai,--- , a n _ x }| + |{J: J~ K, J C {ai,-», a„_ 2 ,a n }| 
-|{J : J ~ K,J C {«!, • • • ,a„_ 2 }| 
(re-|if|)! (n-|ir|-l)! 

rii>i^! (ni-l)!n<>2n»! 
[2(n- -ni](re- - 1)! 

rii>i«i! 

On page 11 of Carter |J, it is proved that ^^w^ )| = J TT^n 7 ! ' P ro P os ition 2.6 of Orlik and 

Solomon § states that x(£ F " (W H *) = (* - l)(t - 3) • • • (t - (2(re - - 3)(t - (re - \K\ +r - 1), 
where r is the number of blocks of size greater than one. Clearly r = n — \K\ —n\. Therefore, 
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{ _ ir -\K\ \W K \ r L Fix{W K ) 1} = ^ | _ _ : 

= \X(K)\. 

Next we proceed to the cases I 2 (m) and G 2 . On page 175 of Orlik and Solomon §§, x(L Fix{Wl<) ,t) 
is computed in the following relevant cases: 

• If n- \K\ = then x(L Fix(Wl<) ,t) = 1. 

• If n- \K\ = 1 then X {L Fix{WK \t) = t - 1. 

• If n — \K\ = 2 then ^(X^'* x (^ s ') ) t) = (t — 1)0 — n), where n + 1 is the number of lines of L 
contained in Fix(Wk)- 

As an application of this computation, we prove the theorem for him) with m even (him) 
with m odd and G 2 are easy and similar). One checks that |A(if)| = 1 for all K contained in the 
two element set II = {01,02}- For K = 0, \n^Wk)\ = 2m and FixiWx) contains m lines of L, so 
the theorem checks. For K such that \K\ = 1, \ N ^} K ) \ = \ and x{L Fix{WK \ t) = t - 1, so the 

theorem checks. Finally, for K = II, | jv]^w^) | and x{L Flx ^ Wli \t) are both equal to 1. 

The proof of the theorem for the exceptional cases -F4, #3, H^, E$, E?, Eg consists of a finite 
number of straightforward calculations based on tables 3-8 of Orlik and Solomon || . We will work 
out the details for an example to illustrate what is involved. 

For the example, take W = E-j and K any subset of II whose Dynkin diagram has type {A{) 2 . 
As the first column of their Table 7 indicates, all such K are equivalent under ~. A glance at 
the Dynkin diagram of Ej thus shows that |A(-£C)| = 15. The third entry in the first row of 
their Table 7 gives that \ n^\w k ) \ = ^' ie end °^ ^ e third row in their Table 7 gives that 

x (L Fix ( w K\t) = (t - 1)0 - 5)0 - 7)0 - 9)0 - 11). This data shows that the theorem checks. 

The only minor complications arise in computing |A(iT)| in cases where there are subsets J with 
Dynkin diagram isomorphic to K, but such that J 9^ K. As the tables of Orlik and Solomon (loc. 
cit.) indicate, this happens only for W = £7, and the values |A(if)| for these cases appear on the 
top of page 279 of their article. □ 

From its statement, it is not evident that Theorem |] generalizes any known identities about 
Coxeter groups. Theorem |2| shows that this is indeed the case. 

Theorem 2 Let W be a finite Coxeter group of rank n with base II. Then 

V ( 1V-W ^ x(L F ^ w «\t) 
jkn \W K \x(L F ^),-l) 

Proof: As is explained on page 274 of Orlik and Solomon ||, elementary properties of Moebius 
functions yield the identity 

J2x(L Y ,t) = e. 



Observe that W acts on the lattice L. Let 0(Fix(Wk)) be the orbit of FixiWx) under this 

\w\ 

\N W (W K )\ 



action. Lemma 3.4 of Orlik and Solomon (loc. cit.) states that \0(Fix(Wfc))\ = w^lw~T\' -^ or 
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Y € L(A), Lemma 3.4 of Orlik and Solomon (loc. cit.) shows that Fix(Wy) = Y. Since the 
parabolic subgroup Wy is conjugate to a standard parabolic subgroup Wk for some K, it follows 
that Y G 0(Fix(W K )) for some K. Then Y G 0(Fix(Wj)) for exactly the |A(if)| many J's such 
that J ~ K. Therefore, the Orlik-Solomon identity becomes 

v \W\ X (L Fix ^\t) = 
^ n \N w (W K )\ \X(K)\ 

The result follows from the formula for |A(-fT)| in Theorem [l]. □ 
Remarks: 

• Observe that setting t = — 1 in Theorem || yields the well-known identity 

As noted in the introduction, this identity has a topological proof and arises in the invariant 
theory of W . It would be desirable to find analogous interpretations for Theorem ^. 

• The identity 

E { _ 1) \K l m 1 = 1 

kn \Wk\ 

has two known generalizations. To describe the first, let di(W) be the degrees of a Coxeter 
group W. Then from Sections 1.11 and 3.15 of Humphreys |J, 

Ken i=i 11 i=i 1 1 

To describe the second, note that W acts on the left cosets vWk by left multiplication. Let 
Jk{w) be the number of left cosets of Wk fixed by w. Let det{w) be the determinant of w in 
its action on V. Proposition 1.16 of Humphreys || states that 

Y,(-l) lKl fK(w)=det(w). 
Ken 

What is the relation of Theorem |2| with these generalizations? Can Theorem [2] be further 
extended to include them? 
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